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FOREWORD 



Assuring quality and promoting appropriate accountability in America's colleges and universities 
are important roles for government, faculty and the institutions themselves. Quality and 
accountability also are important aspects of the growing public concern about education at all 
levels, in particular for students, parents, employers and communities who look to higher 
education with high expectations. In this context, it is entirely appropriate for the Education 
Commission of the States (ECS) to engage policymakers, along with educators, accrediting 
associations and others, in an examination of how quality is defined, what roles various groups 
and organizations play in quality monitoring and improvement, and how these roles can be 
coordinated to create a more effective, less onerous accountability system. This is precisely the 
purpose of this paper, as well as the series of initiatives from which this paper comes and to 
which it contributes. 

As a state membership organization established to work with state policymakers, educators, the 
federal government and other relevant parties, ECS is well-placed to gather facts, facilitate 
important dialogues and help find solutions to difficult policy-related issues that extend across 
state and organizational boundaries. Such are the challenges of dealing with quality and 
accountability in higher education, and ECS has been working on these topics for many years. 
Our involvement in this project began in June 1994, when we gathered policymakers, educators 
and organization leaders at the Wingspread Center in Racine, Wisconsin, to examine what 
quality in higher education means and what roles states, accrediting bodies and federal agencies 
play in monitoring and enhancing attributes of quality. 

One outcome of that initial dialogue involved additional fact-finding through a series of focus 
groups held in six locations around the country, several of which were conducted in collaboration 
with the regional institutional accrediting associations. The information gathered was, in turn, fed 
back to the National Policy Board on Higher Education Institutional Accreditation, an interim 
body charged with recommending changes to existing practices of accreditation. 

A second outcome was a much more public initiative. Many of these same issues and much of 
the information were incorporated into the year-long agenda of 1994-95 ECS Chairman Roy 
Romer, governor of Colorado. This agenda stimulated further dialogue, regional meetings and 
other initiatives and culminated in the widely disseminated report, Making Quality Count in 
Undergraduate Education. While the concern about quality and quality enhancement in higher 
education comes from many sources, this document provides the important and often missing 
perspectives of policymakers, students, employers and other important constituents. 

This paper, Refashioning Accountability: Toward a "Coordinated" System of Quality Assurance 
in Higher Education, is in many ways a third outcome of the 1994 Wingspread meeting. Many 
participants saw the need for an agenda-setting paper that outlines potential contributions of 
current stakeholders in a substantially redesigned accountability system. Accordingly, ECS 
convened two follow-up meetings, in February 1996 and February 1997, on the premise that 
much improvement in existing roles and practices could be accomplished on a cooperative basis. 
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Participants in these meetings were drawn from a wide range of people involved in higher 
education on various levels, including the federal government, state governments, regional and 
specialized accreditation agencies, and major national higher education associations. As the 
paper suggests, there is room for others as well; the tasks as well as the paper are very much 
works-in-progress. At the same time, participants agreed that sufficient consensus already has 
emerged from our discussions to warrant distribution of the document as a working paper. We 
intend to initiate efforts to further develop the paper's core concepts and to begin collaborative 
work on a few action agendas that all agreed were important. 

ECS will continue to act as convener for further initiatives designed to flesh out these parallel 
lines of work. At a minimum, these include: 

• Develop a process to negotiate and ratify multi-lateral cooperative agreements. How 
should people come together? What would a national coalition for creating a refashioned 
system of quality assurance look like? What are the core issues? 

• Identify clearly specified initiatives that participants agree to begin work on immediately, 
and that will begin the process of creating such a system. 

Both activities will become especially salient as the reauthorization process for the Federal 
Higher Education Act proceeds in the coming year. The visibility of the reauthorization process, 
coupled with its importance to the higher education community, will in itself provide an 
important opportunity for collective action. 

Participants in the 1996 and 1997 meetings also agreed that collaboration on information-sharing 
and the establishment of national definitions for reporting information about institutional 
performance represented important areas where progress can be made immediately. Both 
activities, it should be stressed, need cooperative rather than regulatory approaches. Sufficient 
voluntary adoption of common reporting and information-sharing conventions can help launch 
and sustain a more thoroughgoing refashioning of higher education's quality enhancement and 
accountability systems. Cynics may remind us that such efforts have been made before to no 
avail. While the task will be difficult, we believe major changes in higher education's operating 
environment — including most prominently politics, economics and technology — when 
coupled with a new spirit of cooperative effort among key accountability players will prove 
sufficiently synergistic for these efforts to make a significant difference. 

We are under no illusions regarding the challenges involved in this effort. But we believe it to be 
important and we welcome the participation of all. 

Kay McClenney Charles S. Lenth 

Vice President ECS Director of Policy Studies for 

Education Commission of the States (ECS) Higher Education 
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REFASHIONING ACCOUNTABILITY: 
TOWARD A "COORDINATED" SYSTEM OF QUALITY 
ASSURANCE FOR HIGHER EDUCATION 



Peter T. Ewell 
Jane V. Wellman 
Karen Paulson 



INTRODUCTION AND OVERVIEW 

This paper critically explores the current "architecture" of higher education accountability in the 
United States and considers more specifically how this design might be improved. Its main goal 
is to define pertinent issues surrounding quality assurance and accountability with sufficient 
clarity to invite further conversation among the relevant parties — the federal government, state 
governments, higher education boards, accrediting associations, professional associations and 
others. Interaction and collaboration among these parties will determine whether consensus can 
be reached about particular problem areas and strategies for improvement. 

Effective realignment demands an approach to accountability that involves multiple entities 
operating for the most part independently of one another in the context of a largely 
non-regulatory policy environment. The coordinated approach outlined is designed explicitly to 
capitalize on the unique strengths and perspectives of different parties-at-interest, to increase 
collaboration among them and to reduce excessive duplication of efforts. Such a quality 
assurance and accountability system also is intended to support and work in conjunction with 
faculty and campus-based efforts to improve institutional quality. 

This paper grew out of a February 1996 meeting of higher education policy leaders convened 
jointly by the Education Commission of the States (ECS) and the six regional accrediting 
associations. The sponsors shared two main convictions. First, they believed that important 
accountability-related problems continue to trouble American higher education and that a new 
look at the topic might help address these problems. Second, and at least as important, they were 
convinced that inadequate communication among key parties was exacerbating accountability 
problems. 

Conversations during the meeting confirmed the sponsors' conviction that new thinking about 
higher education accountability is imperative and that exploring some possible new architectures 
of accountability was worth undertaking. As a result, Peter Ewell of the National Center for 
Higher Education Management Systems (NCHEMS) and Jane Wellman of the Institute for 
Higher Education Policy prepared a working document to build on this conversation, to establish 
an analytic framework for discussing the topic further, and to advance some specific suggestions 
about how a refashioned accountability approach could evolve. Participants in the original 
meeting reviewed and critiqued the resulting document, which was re-written to incorporate 
reviewer suggestions with the assistance of Karen Paulson of NCHEMS. 
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A second draft was distributed for review prior to a follow-up meeting convened by ECS in 
February 1997. Participants in the second meeting strongly supported the conceptual approach 
outlined in the paper and suggested that ECS publish it for wide dissemination and discussion 
by the participating organizations. 

The paper's first section briefly discusses the emergence of accountability as a policy issue 
and advances some basic principles for evolving workable solutions to this issue in the current 
policy climate. Specific aspects of the "accountability problem" are examined in greater detail, 
leading to the conclusion that there is not one such problem but many, not all of which can be 
addressed through the same policy mechanism. 

The second section looks at what is meant by accountability from an analytical standpoint and 
examines specific attributes of the proposed approach. An overview of the specific interests of 
particular actors in the accountability conversation and what each of them is especially good 
(and not so good) at doing follows. The concluding parts of this section extend the discussion 
by advancing some tentative assignments for participants in a refashioned system of quality 
assurance. 
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BASIC PROBLEMS AND GROUND RULES 



Why Address Accountability (Again)? 

In recent years, discussions about accountability for higher education have become both common 
and contentious. Many reasons for this have been advanced, but virtually all can be related to two 
factors: declining public resources and the sense that colleges and universities are ill-prepared to 
meet needs of the 21st century. Policy leaders at both the state and federal levels have struggled 
to improve higher education's responsiveness to these conditions through a range of increasingly 
proactive initiatives. But they have experienced growing frustration about the inability of their 
actions to change the ways colleges and universities behave. A new political climate also has 
emerged emphasizing deregulation and government down-sizing. Both conditions have 
decisively shaped the way policymakers approach conversations about accountability. 

Struggling hard to make ends meet, institutional leaders meanwhile are in growing rebellion 
against what they perceive as overly bureaucratic approaches to achieving accountability. At the 
most basic level, they see complex, overlapping and duplicative systems of reporting and 
assessment that are unnecessarily expensive in terms of time and resources. In addition, they 
view with alarm a growing shift away from established institution-centered measures such as 
accreditation toward government-mandated, standards-driven accountability systems they believe 
threaten institutional diversity and autonomy. Protecting these perceived historic virtues of 
American higher education is the principal agenda they bring to the accountability table. 

Debates along these lines crystallized most visibly in recent years around a set of federal 
initiatives intended to create a more comprehensive national approach to accountability. In 
particular, the 1992 Amendments to the federal Higher Education Act mandated a number of new 
relationships among the federal government, states and institutional accreditors. Specifically, the 
1992 Amendments had three controversial features: (a) creation of State Postsecondary Review 
Entities (SPREs), (b) a significant extension of federal regulatory control over the process of 
voluntary institutional accreditation (including extensive prescriptions about the particular 
quality standards and review mechanisms accreditors must use in examining institutional 
performance), and (c) strengthened federal oversight of institutions with respect to established 
standards for administrative and financial responsibility (including more frequent reporting of 
information and more proactive monitoring of performance). All three initiatives were stimulated 
by visible problems in federal student assistance programs that were becoming more and more 
difficult to ignore. More broadly, they were premised on the use of federal aid eligibility as a 
lever for deeper reforms in higher education. 

The SPRE provisions were opposed with unusual vigor by the higher education community as an 
unwarranted trespass on institutional autonomy. At the same time, the threat of a federal takeover 
of accreditation stimulated a highly visible effort to strengthen the existing system of 
nongovernmental self-regulation. Immediately after the regulation process for the 1992 
Amendments began, the six regional accrediting bodies joined with the major Washington-based 
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institutional associations to create the National. Policy Board on Higher Education Institutional 
Accreditation (referred to as the NPB). 

This body, in turn, proposed the formation of a new national organization designed to strengthen 
institutional accreditation — the traditional academic mechanism for self-regulation — both to 
preserve it from political interference and to address perceived weaknesses in the existing 
system. Among the most prominent features of this new organization was a governing board that 
contained a majority of lay members and a multi-tiered structure of institutional recognition. 
Somewhat to the surprise of its creators, this new body also was vigorously opposed. As a result, 
its initial design for an integrated national approach to accreditation was replaced by a body 
controlled by a majority of college and university presidents that eventually became 
institutionalized in the form of the Council for Higher Education Accreditation (CHEA). 

Federal authorities also changed the course of their own regulatory and oversight initiatives. This 
occurred partly in response to vigorous and organized institutional opposition. But more 
important, it reflected a significant change in political climate signaled by the 1994 elections. In 
the wake of a "deregulatory" sweep in these elections, SPRE was defunded and further federal 
efforts to nationalize accreditation were largely abandoned. Instead, federal authorities decided to 
review comprehensively their core responsibilities for institutional oversight and, in particular, 
signaled their intention to focus regulatory attention on only a few "problem" institutions. This 
abrupt change of federal policy further eroded support for the NPB initiative, which most in the 
academic community felt was no longer needed. 

This turbulent history has convinced many members of both the policy and academic 
communities that the entire accountability system needs realignment. Even as the regulatory 
mechanisms established by the 1992 Amendments were being rolled back, both academics and 
policymakers realized the problems stimulating these initiatives had not gone away. Challenges 
such as rising costs in higher education, the continuing operation of diploma mills that defraud 
both students and taxpayers, the inadequacy of current quality assurance processes when applied 
to distance learning and other technology-intensive learning environments, and the continuing 
estrangement between academic and political leaders pose immediate political risks if they 
remain unattended. More important, they threaten the long-term integrity and viability of the 
postsecondary enterprise as a whole. 

SPRE and the NPB both left the scene before they had a chance to do anything, leaving 
unanswered the question of whether agencies of this kind might have been able to address higher 
education's many accountability problems. The experiences of both these nascent organizations, 
though, demonstrated decisively that new regulatory initiatives stand little chance of getting 
started under current political conditions. For those persons seeking to reform the system, the 
heart of the resulting dilemma can be easily stated but not so easily resolved. On the one hand, 
there is simply no political support for achieving accountability through centralized means, 
whether coordinated between the states and the federal government or undertaken by the federal 
government itself. Reformers must seek to define a new basis for political support. On the other 
hand, no one has a real conception of what a system of accountability other than one based 
on regulatory control ought to look like. To be effective, future action to consider alternative 
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structures must explicitly face this dilemma. Reform must start with a concerted effort to reach 
conceptual agreement on the principles and structure for such a system. 

Ground Rules for Appropriate Action 

Any accountability system requires the understanding and support of the actors that comprise it. 
Beneath the apparent solidity of the current array of accountability-related statutes and 
enforcement mechanisms is the reality that such regulation relies primarily on the good will and 
cooperation of the regulated parties. This is true especially for the key elements of peer review 
and institutional data collection on which higher education's approach to accountability depends. 
Insofar as the system works at all, it does so because all parties "buy in." Without similar 
understanding and consensus for a new approach, people simply will ignore a new round of 
mandates and nothing will happen. 

To avoid this outcome, and the equally undesirable result of unproductive contentiousness, a 
clear set of operational ground rules is required. Among the principles that seem most useful in 
this regard are the following: 

1. Do no harm. The existing structure of accountability and quality assurance in higher 
education is both intricate and familiar. Tinkering with the existing system with no clear 
policy objective in mind — as arguably, the implementation of the 1992 Amendments 
demonstrates — may only complicate matters without improving conditions. Any 
alternatives considered, therefore, should build on the familiar and be carefully examined 
for potential side effects that might damage areas where the current approach functions 
well. 

2. Define particular problems and match them with particular solutions. Fashioning 
specific accountability policies to meet clearly defined problems also appears to be a 
better approach than simply multiplying initiatives. Pervasive pressures to improve 
accountability frequently stimulate a deluge of data-collection and institutional-oversight 
initiatives, often sacrificing real purpose for the appearance of activity. Clearly, there are 
some problems for which regulatory solutions are both reasonable and appropriate — 
detecting fraud and abuse, punishing malfeasance, or enforcing consistent information 
collection and reporting, for example. Others — such as students failing to learn or being 
unhappy when they are at college — are not likely to be fixed through accountability 
systems, whatever their form. A comprehensive accountability system should be 
consciously designed to distinguish between those problems that really can be addressed 
by policy and those that cannot. 

3. One size does not fit all. A predominant theme in the current approach to accountability 
is a tendency to treat all institutions similarly, whatever their individual circumstances. 
While arguably more equitable, this approach ignores important differences in context 
and the fact that different kinds of accountability problems tend to be associated with 
different kinds of institutions. Conducting mandatory unannounced site visits to 
institutions, for example, in most cases yields very little in terms of real information. 
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While such visits may be useful tools when hard evidence of fraudulent activity exists, 
they do not need to be mandatory for all institutions. 

4. Avoid unnecessary duplication. There is no compelling reason for the actions of 
different accountability players to duplicate one another. Currently, the federal 
government, the states and the various accrediting bodies all collect similar information. 
Not only do performance standards need to be different for institutions with different 
missions, but the means to evaluate and enforce such standards also should differ for 
specific agencies — largely because it is in the basic interests of these agencies to be 
concerned about different things. 

5. Reserve for government those activities that only government can do. All parties must 
recognize from the outset that specific aspects of higher education are, and should be, 
regulated. The heavy use of public funds, issues associated with consumer protection and 
"ability to benefit," and the belief that education is a key national asset all require some 
measure of public scrutiny of how higher education does its business. But regulatory, and 
indeed, governmental, approaches are the most efficient means for addressing only a 
subset of what we have come to call "accountability problems." Many problems being 
addressed through accountability are not really amenable to government solution at all. 

6. Reserve for colleges and universities (and the academic community) those activities 
that only higher education can do. Equally important is the fact that academic 
peer-review processes are unique in their ability to do some things credibly and 
effectively. A 1995 report, Accountability of Colleges and universities: An Essay, argued 
convincingly that the internal assessment and evaluation processes at the institutional 
level — if done seriously and well — are important ingredients in fashioning a new 
national approach to higher education accountability. External requirements for 
accountability should to a considerable extent be built upon, but above all not do damage 
to, these internal systems. 

7. Speak plainly. Partly because of the academic love of complexity, and partly to maintain 
a useful degree of ambiguity as well, academics tend to raise deep existential issues about 
the nature of learning and the autonomy of scholarship when the topics of accountability 
and assessment are raised. The resulting discussions are largely impenetrable to others. 
Government officials — especially those in the regulatory branches — have an equal gift 
for impenetrability. The net result can be a kind of "white noise" that leads policymakers 
and commentators to the understandable conclusion that higher education "just doesn't 
want to be accountable." Whatever the shape of an eventual new system of quality 
assurance, this perception must be avoided. For both parties, the principles on which a 
new approach is founded should be clear, and the language in which its components are 
expressed should be unambiguous. 

Principles such as these need to be agreed upon from the outset as a prerequisite for evolving any 

new accountability approach. At the same time, throughout its implementation, such principles 
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can be used as a constant check to ensure that the resulting system remains true to its original 
purposes. 

What Problems Are We Trying To Solve? 

The February 1996 meeting in Denver began with a brainstorming exercise designed to list all 
problems — perceived or real — that participants thought a redesigned accountability system 
might help address. The resulting array of problems is interesting because of both its length and 
its span of topics. They range from broad problems such as eroding public financial support for 
higher education, through mid-range issues such as the difficulty of measuring quality for 
distance-learning programs, to specific problems such as student loan default rates. The size, 
complexity and lack of focus in the accountability system itself — and in particular, in the triad 
of oversight shared by the federal government, accreditors and the states — was also a source of 
concern. 

Without according particular priority to any one of these problems, and without editorializing 
about which are amenable to solution through policy, it is useful to present them from the general 
to the specific. This section, therefore, begins with a discussion of problems related to global 
aspects of the relationship between higher education and society, proceeds to questions about the 
design of the accountability system and finally examines some specific public policy problems. 

Global Social and Economic Trends 



Overall demands for accountability. Heightened public expectations about accountability 
and performance affect all aspects of society. No major industry or service area — from 
automobile manufacturing to medical care — remains untouched by this trend. At the same 
time, higher education serves an increasingly sophisticated and consumer-oriented public, 
one that expects to be able to "buy performance and get results." There is less public 
deference to the norms of professionalism and expertise in all professional fields and lower 
tolerance for soft regulatory systems that do not guarantee results. Along with this come 
greater demands for empirical and "rational" structures of choice, whether public or 
individual. Choices that in the past were treated as matters of values and taste — such as 
which college to attend or what lines of research faculty should pursue — are now expected 
to be rationally explicable and analytically defensible. 

Shifting views of the role of government. Principles guiding public policy also appear to 
be in the midst of major change. Definitions of what constitutes the public interest and 
conceptions of the appropriate role and scope of government that have prevailed since the 
New Deal are giving way to different notions of private rights and public responsibilities. 
The appropriate roles of the federal government vis-a-vis state governments and of the 
private sector in setting national standards are under active debate in virtually every public 
policy area, including the environment, health care, welfare, nutrition, transportation and 
education. As yet, there is no consistent message in this debate as it affects higher 
education. In fact, two forces seem to be operating simultaneously — the press to deregulate 
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and privatize the finance system on the one hand, and the press to raise academic standards 
and improve performance on the other. 

• Private economic benefits resulting from education investments. Concerns are growing 
about whether the private costs of higher education (both in terms of tuition and foregone 
income resulting from excessive time to earn a degree) are really worth it. Rising costs, on 
the one hand, and the belief that access to higher education is a personal economic 
necessity, on the other hand, drive these concerns. Arguments prevalent in the 1980s which 
suggested that education has a public and a social value leading to increased collective 
benefits for society are being joined by the strongly held belief that higher education confers 
a largely private, primarily economic value for individuals. Indeed, the decision to attend 
college increasingly is seen as an individual investment decision, with an expectation that 
this investment must pay off personally. This argument weakens the case for public support 
for higher education and raises the salience of consumer protection as a major 
accountability issue. 

• Declining public financing for higher education. At both the federal and state levels, the 
general tax revenues available to support higher education are threatened. Higher education 
funding remains one of the most vulnerable components of each state's discretionary 
budget. Funding for colleges and universities tends to be what is left over after allocation 
decisions for entitlement programs, health care and prisons already have been made. Federal 
and state officials face increasingly difficult decisions about who gets what leftovers, 
decisions that all too often also involve competition between higher education and the 
public schools. 

• Elementary and secondary school performance. Issues about student performance in 
elementary and secondary schools relate both substantively and politically to the 
accountability debate in higher education. The many connections between postsecondary 
and K-12 education include the relationship between college admission requirements and 
exit-level high school achievement, the quality of teacher preparation in colleges, and the 
sense of common purpose (or lack thereof) that exists between colleges and schools. As 
public schools are perceived to be increasingly at risk, the responsibility of the collegiate 
community to share some ownership of the problem becomes a major issue for public 
officials. As the standards movement in elementary and secondary education becomes an 
important political linchpin of the school reform movement nationwide, its strongest 
advocates believe the same political and reform logic also applies to higher education: 
stronger standards stimulate reform, which, in turn, creates closer connections between 
colleges and the public schools. Following this logic, standards-driven reform for higher 
education — and the performance indicators that underlie it — have become hot topics on 
state policy agendas. 

Problems with the Design of Higher Education's Accountability System 

• Dissatisfaction with accepted definitions of "excellence" in higher education. Norms 
for measuring quality in higher education that have prevailed for the last half century are 
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now showing visible signs of wear. Traditional measures of institutional excellence have 
centered on the selectivity of undergraduate admission in the form of high school grades 
and test scores, and the presence of graduate education and research. Such conventional 
measures of merit are under intense scrutiny, in part because of a highly visible assault on 
the college admission process in connection with recent legal and political challenges to 
affirmative action. At the same time, the legitimacy of research and graduate education as 
the acme of institutional excellence is being questioned in virtually all political quarters. As 
a consequence, alternative measures are being called for that define institutional quality in 
terms of outcomes and value-added paybacks. 

Inadequate links between internal assessment systems and external accountability. 

The assessment movement in American higher education took hold in the mid-1980s in part 
because its proponents argued that the internal value of self-evaluation for 
self-improvement would be lost if its results were used to compare institutions for external 
purposes. Their presumption was that self-evaluation could be used most effectively to aid 
in institutional improvement, an objective which would be more harmed than helped by 
external comparisons. Much of what institutions evolved to evaluate themselves during this 
period, therefore, never was designed to yield public information or to support summative 
judgments about quality. Furthermore, the extent to which institutions actually have 
implemented such processes — whatever their purposes — is at best uneven. The weak link 
between internal assessment and external accountability is especially problematic for 
accreditation because this process attempts to play three distinct roles simultaneously: 
certification of quality, institutional improvement of quality and public communication 
about quality. More and more, doubts are being raised about whether a single process can 
do all these things effectively. 

Poor communication and inadequate role differentiation among the various 
accountability players. The current accountability system contains many actors and has 
grown incrementally over time. At a minimum, its players include institutional and 
system-level governing boards, faculty, administrators, accreditors, disciplines and their 
associations, state policymakers and agencies, the federal government and the market. 
Moreover, over time, the roles and responsibilities of the various actors have shifted as 
programs and policies have changed, or in response to specific legislative initiatives. For 
public policy purposes, members of the traditional accountability triad for higher education 
— states, institutional accreditors and the federal government — have always been accorded 
priority. Yet, the respective roles and responsibilities of these three entities have never 
really been harmonized, except perhaps in the realm of federal financial aid. Even in this 
case, there are overlapping responsibilities and expectations, resulting in redundant review 
and information-collection processes. There is also a history of poor communication among 
members of the triad, both in terms of sharing data about institutions and of collaborating 
professionally. 

Uneven and/or poorly enforced state licensure standards. As a 1994 State Higher 
Education Executive Officers (SHEEO) report observed, studies of state licensure in 
postsecondary education repeatedly have revealed two problems with such mechanisms: 
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weak standards and poor enforcement of these standards. State licensure is the first hurdle 
an institution must pass to open its doors. Accreditors require all institutions to be licensed 
by the state before applying for institutional accreditation. In a number of states, however, 
the criteria for licensure are weak and have more to do with building safety or code 
compliance, for instance, than with educational quality. 

In many states, institutions that are already accredited do not need to undergo further review 
for licensure, and enforcement of state standards is often uneven. In most states, the 
licensure function is supported by federal Veterans Administration program-review funds, 
supplemented by fees collected from private institutions that are not accredited. This 
practice provides a very small revenue base for oversight and also creates a disincentive for 
the state to remove licenses from institutions since this would further dilute the available 
revenue. Finally, many states view "gatekeeping" as a one-shot function and do not conduct 
ongoing institutional reviews after the colleges are licensed. 

Eroding confidence in the role of institutional accreditation. In political circles, 
confidence is eroding about the effectiveness of institutional accreditation in the system of 
accountability. In the current structure, accreditation plays both internal and external roles, 
as it claims the dual functions of assessing quality and of communicating results to the 
public. Accreditation also has been accorded a quasi-regulatory function because it sets and 
enforces minimum institutional standards for state licensure and serves as a gatekeeper for 
federal Title IV eligibility. The effectiveness of accreditation as a quality assurance device 
is being questioned in part because of the widespread perception that its review mechanisms 
continue to rely largely on input-driven measures of institutional resources to certify quality 
instead of value-added or output measures. 

At the same time, accreditation's effectiveness as a public partner in the gatekeeping process 
is being criticized because institutional accreditors have not been aggressive in monitoring 
compliance with federal regulations and in pursuing institutions in shaky financial 
situations. Finally, the information that accreditors make available to education consumers 
— parents, students, state policymakers and the federal government — is seen as much less 
valuable by many parties than that which is obtainable either from government or 
third-party providers such as the annual consumer guides to colleges published by national 
magazines. 

Specialized accreditation and state licensure. For many professions and occupations, 
there is a strong link between specialized accreditation and state licensure. Some 
professional associations consistently pursue a state legislative agenda that requires 
candidates for professional licensure to graduate from programs that also have their stamp 
of approval in the form of accreditation. While this agenda may be justified on grounds of 
quality, it might also be perceived as restraint of trade. Although some specialized 
accreditors have revamped their review process to focus more specifically on value-added 
and results or outcomes rather than processes or inputs, others have not. Some college 
presidents express frustration when they perceive specialized accreditation to be duplicative 
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or wasteful or — at its worst — a form of blackmail that requires resources to be put into 
programs that cannot be justified on the grounds of institutional mission. 

• Litigation and enforcement. The challenges of using litigation — and the more general 
difficulty of making harsh decisions stick — continues to plague state officials, accreditors 
and the federal government. This issue is not just attributable to litigious institutions that 
are, after all, pursuing their own self-interest and exercising established constitutional 
rights. It appears to be increasingly difficult to translate a system of quality enforcement that 
relies primarily on qualitative judgments backed by impressionistic data into a set of clear 
standards that are defensible in court. This situation again raises the question of whether a 
system primarily designed to improve quality also can be effective in enforcing standards. 

• Performance of proprietary and vocational institutions. Underlying much of the concern 
for accountability in higher education is the question of whether the same performance 
expectations and measures of achievement should apply equally to vocational and 
proprietary schools and to degree-granting, public and nonprofit institutions. In the financial 
aid arena, most of the institutions that have run into trouble with the federal government — 
both as measured by student loan default rates and institutional closures — have been 
for-profit trade schools. Congress, however, has not been willing to set specific standards 
for performance that apply only to proprietary and vocational institutions. 

• Increased reliance on "market forces" but little consensus about what these forces are 
and their specific roles. The particular role of the marketplace and how government policy 
should relate to market-provided incentives has been largely unexamined in discussions of 
higher education accountability. As government roles shrink and pressures for increased 
accountability expand, a more explicit understanding of what role the marketplace might 
play is increasingly important. Left to itself, the market is not likely to act as an "invisible 
hand" that regulates equally well in all situations. For the marketplace to work optimally, 
moreover, consumers need high-quality, widely available information. As a result, some 
kind of government or self-regulatory function that guarantees adequate consumer 
information seems an essential condition for the market to work as a quality-control 
mechanism. 

Specific Public Policy Problems 

• Diploma mills. Large numbers of diploma mills continue to operate around the fringes of 
state licensure and frequently claim accreditation from phony accreditors. These institutions 
prey on the most vulnerable student populations, students who need diplomas to get decent 
jobs, but who lack the sophistication to penetrate the maze of conflicting information about 
what constitutes a reputable institution. This situation is exacerbated by the increasing use 
of distance-learning and telecommunications technologies, which have become the delivery 
mechanism of choice for such institutions. State licensure and accreditation systems are 
poorly equipped to address these concerns, even when they are aware of them. Not only are 
the geographic distinctions that define jurisdictions increasingly irrelevant in such cases, but 
standards of evidence and criteria forjudging performance intended for conventional 
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degree-oriented institutions are increasingly unable to separate the reputable "virtual 
university" from the "emperor's-new-clothes" university. 

• No standards for distance-learning programs. Conventional "quality" measures in higher 
education tend to assess quality in terms of inputs — particularly the number of student 
credit hours and the amount of faculty contact with students — rather than the amount of 
learning that has occurred. This approach to measurement increasingly is a public policy 
problem because of the growth of distance learning. As state officials want to maintain 
access in higher education, they increasingly are looking to distance learning as a way to 
stretch limited instructional budgets. Absent a measure of actual learning as a replacement 
for the traditional student credit-hour measure, there are no consistent means for evaluating 
such programs to determine if their quality is at least comparable to that of campus-based 
programs. 

• Student-loan defaults. The student-loan default problem, and the cost of defaulted loans to 
the federal government, was widely perceived to be the underlying cause of the federal drive 
to strengthen accountability in 1990-92. Since then, the net cost of student-loan defaults to 
the federal government has been reduced dramatically, decreasing from $2.7 billion in 1991 
to an estimated $400 million in 1995. Despite this decline, the student-loan default issue, 
coupled with the rising dependence of students on loan funds to finance their education, 
means that it will continue to be raised as a public policy issue. 

• Fraud in the Title IV programs. Fraud occurs when institutions submit false 
documentation about students to establish institutional eligibility to participate in the Pell 
Grant program, or when institutions enroll students in phony programs in order to receive 
loan checks. The institutions close, and students are left without any education or hope of 
getting a job to repay their loans. The cost of institutional fraud in federal Title IV programs 
is not significant when compared to the cost of student loan defaults, and the close 
relationship between fraud and student-loan default has caused some to equate the two. But 
they remain different problems and require quite different solutions. 

• Rising tuition costs. Rising tuition cost is becoming a major public policy issue as well as a 
growing public-relations problem for higher education. In just the last decade, tuition at 
private four-year colleges increased overall by 95%; at public four-year colleges the 
increase was 82%. The rising cost of college has received considerable recent media 
attention, for example, in The Philadelphia Inquirer, Newsweek, The New York Times and 
on the television program, 48 Hours. Congress responded in 1996 by initiating oversight 
hearings on the rising costs of college. 

While many of the problems outlined above and summarized in Figure 1 are related, each is 
sufficiently distinct that it must be addressed on its own terms. Each also involves most directly a 
particular set of parties, as outlined in the following section. Another primary test of any 
proposed new approach, therefore, is the degree to which it allows these parties to get a better 
policy handle on some subset of this array of issues. 



ECS! Refashioning Accountability/Page 12 



FIGURE 1 
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TOWARD A ’’COORDINATED” SYSTEM 
OF QUALITY ASSURANCE 



In the current policy environment, any approach to addressing the many challenges posed in the 
previous section must rely on the coordinated, but essentially independent, actions of many 
players. Postsecondary education has grown sufficiently complex that a single accountability 
approach applied to all the functions of all types of institutions for all audiences is both 
inappropriate and infeasible. There also is no political consensus to support a centralized 
quality-control system, as the fates of the SPRE and NPB efforts have shown. 

One alternative is the conscious evolution of what might be explicitly termed a "coordinated" 
national system of quality assurance. Such a system has two core attributes. First, it rests 
fundamentally on the basic interests of each of its players. This feature acknowledges explicitly 
what already has been demonstrated historically — that individual actors within such a system 
cannot be expected to invest seriously in and discharge accountability functions not related to 
their own core missions or which do not pay off for them directly. As a corollary, this attribute 
accounts for (and indeed, should be designed around) the particular strengths and limits of the 
various actors. Second, such a system relies far more heavily on information and market 
mechanisms than on the use of direct regulation and inspection. This feature also reflects the 
current political environment, while at the same time capitalizing on what is known about 
institutional responses to incentives. 

A coordinated approach of this kind differs fundamentally from both the duplicative mechanisms 
of the traditional triad and their far more deliberate counterparts established through the 1992 
Amendments. Unlike both approaches, it relies on ongoing coordination and shared information 
within the framework of a clearer division of labor among the various parties-at-interest. At the 
same time, it recognizes the fact that the quality-assurance processes operated by different 
entities have legitimately different purposes and clients and may, therefore, look appropriately at 
quite different things and employ quite different standards. Finally, it recognizes the possibility 
that the universe of useful quality-assurance players may include more than just the triad's 
established array of accreditors, states and the federal government. 

Evolving such an approach demands greater conceptual clarity in at least two areas. First, it 
requires explicit recognition that many motives and strategies are entangled in current 
quality-assurance practices. Second, it demands a better understanding of the specific strengths 
and limits of the individual actors that engage in these practices. 

Purposes and Strategies 

One lesson that emerges from over a decade of state-level policy experience with assessment is 
the importance of being clear about motives. Like accreditation, most assessment policies began 
with the assumption that a single, institution-centered process could serve as both a credible 
external accountability mechanism and as a guide to institutional improvement. Increasingly, 
however, state officials recognize that the same process cannot effectively discharge both 
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functions. Not only do they require different kinds of data, but they also have fundamentally 
different audiences. 

Cutting across the resulting difference in purposes, moreover, are a variety of strategies for 
accomplishing these purposes, including direct regulation, concrete incentives (including relief 
from direct regulation) and the use of information. Being clear about what the potential elements 
of a "coordinated" system of quality assurance might look like for each of these dimensions is an 
essential prerequisite for further development. 

Among the specific purposes that might be considered are: 

• General public accountability. This purpose embraces both direct accountability for 
expending public funds and the overall returns on this public investment. Its primary 
audience consists of policymakers and the general public. As a result, the information 
provided must be clearly accessible and widely available. While it need not involve explicit 
comparisons among institutions, the process should at minimum allow summative 
judgments to be made about "cost effectiveness" and the current "condition of the asset" in 
ways that communicate directly to a lay audience. A parallel mechanism outside higher 
education might be a corporate stockholders' report. 

• Assuring capacity. This purpose centers on determining whether or not a given institution 
is minimally capable of doing what it claims to do. Basic consumer protection falls within 
this rubric (e.g., most current state licensure and basic candidacy standards for regional 
accreditation), as does certification of an institution to act as an agent for another actor (e.g., 
Title IV "Standards of Institutional Capacity"). Students, employers and other institutions 
are potential clients of individual higher education institutions and persons or agencies 
contracting with institutions to act on their behalf for specific purposes. Again, the 
information required must be public but should be oriented toward the institution's 
compliance with specific minimum standards. Relevant parallels outside higher education 
include compliance with a building code, the certification of a product by Underwriter's 
Laboratory or an independent bond rating. 

• Informing choice. This purpose centers on providing information that will enable higher 
education's various customers to make individual purchasing decisions. While an assurance 
of basic capacity represents a part of this function, it is by no means sufficient information 
to help all customers make good choices. For students and potential students, information 
about expected results and access to key experiences and resources will be required. For 
employers, information about the specific capacities of graduates and their track record in 
the workplace may be needed. Whatever the audience, the form of the information provided 
must be public and presented as a set of comparative performance dimensions of interest to 
a particular client. A non-higher-education parallel might be the product reviews contained 
in Consumer Reports. 

• Informing internal improvement. While all three of the preceding purposes have an 
indirect role in driving individual institutional improvement, this purpose centers explicitly 
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on providing direct guidance for internal improvement. Because the primary, indeed, the 
only, audience is the institution itself, the information must have sufficient detail to inform 
action and must be tailored to fit the individual context and circumstances of each 
institution. In this case, such information should not be public because, in the absence of 
knowledge about context and intended remedial actions, it likely will be misinterpreted. A 
parallel mechanism outside higher education might be a consulting report or the 
"management letter" that typically accompanies results of a financial audit. 

Many of the same basic pieces of information will be relevant across all these purposes, however. 
A refashioned system therefore, might work best if supported by a common set of data 
definitions and reporting conventions on key descriptive or outcomes measures, although this 
approach need not demand completely parallel reporting. Indeed, the problem is not so much that 
existing quality assurance processes are different as they are multi-purpose. Clarifying exactly 
which actor in the system is most capable of accomplishing which purposes, and focusing its 
efforts specifically to meet these purposes, can help in crafting a more effective approach. 

Each of these purposes, in turn, can be furthered using more than one policy strategy. Among the 
most prominent strategies available are the following: 

• Direct regulation. When applied to matters of accountability, this strategy rests 
fundamentally on the establishment and enforcement of unambiguous standards. Key words 
here are standards, unambiguous and enforcement. Some formal standard is required for 
regulation to occur at all. But because the basic logic of regulation is to prevent bad things 
from happening, the natural tendency is for standards to become increasingly elaborated as 
new bad things are uncovered. Moreover, as standards are applied to non-quantitative, 
multi -dimensional or client-specific domains ("academic quality" as opposed to "whether 
the institution provides accurate information in its catalog," for instance), regulatory 
compliance becomes increasingly less workable. Both dynamics have led to a good deal of 
linguistic confusion about what is really meant when standards are discussed. 

On the one hand, there is a clear need for minimum threshold levels of basic capacity, 
applicable to such functions as recognizing eligibility, that may be furthered through direct 
regulation. On the other, there is an emerging drive to establish standards about outcomes or 
performance — standards more applicable to functions such as informing choice or forcing 
improvement and less amenable to direct regulation. Finally, active enforcement (and the 
sanctions that go with it) is what really defines a standard in the first place: lack of it means 
there is no standard, while differential application means that different standards apply to 
different circumstances, a fact that ought to be recognized up front. 

• Incentive systems. Applied to matters of accountability, this strategy rests on the ability to 
reward performance either directly through revenue or indirectly through enhanced 
recognition or regulatory relief. In the first case, the strategy is available only to agencies 
such as state and federal governments that provide substantial resources to institutions. In 
this arena, approaches such as performance funding, incentive grants or contractual 
arrangements in which institutions receive additional funds for meeting particular 
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conditions have been widely tested at the state level. More common are indirect incentives 
that enhance an institution's status and recognition, thereby enhancing its ability to raise 
revenue or effect savings. 

To be effective, recognition and revenue-raising incentives can be given only to a few 
institutions. For example, a Baldridge award or the "Level III" recognition in the original 
three-tiered NPB proposal of 1994 would lose most of its value as an incentive if too widely 
given out. The success of regulatory relief incentives depends on the willingness of the 
agencies that require institutions to report on many accountability measures to substitute 
broad performance indicators for detailed oversight. In a government context, an example 
might be providing institutions with greater flexibility in using funds previously subject to 
strict line-item accounting control. For accreditors, an example might be allowing 
institutions to provide assurances of a broad range of compliance areas where this is 
warranted. In essence, this strategy is used already by regional accreditors in the practice of 
awarding recognition periods of differing lengths to institutions, based on the results of a 
given reaffirmation visit. At the same time, accreditors increasingly are allowing institutions 
to engage in "focused" or "tiered" reviews that use the self-study process to concentrate on 
quality improvement in key areas for institutions for which baseline accreditation is not an 
issue. 

• Information-driven markets. This strategy relies on the mechanism of consumer 

information both to signal the presence of quality and spur its improvement. To be effective 
in the first role, the information provided must relate to real choices that potential customers 
must make about things they think are important. This, in turn, implies that quality 
dimensions be externally defined. As a result, they may center on concerns that have little to 
do with what traditionally has been labeled "academic quality." What actually is provided 
will be far more credible if supplied by third parties substantially outside the institution's 
influence or control. Government agencies may have a part to play in supplying such 
information, but in other industries this is typically an independent function. To be effective 
in the second role, institutional officials must believe the information can help them 
improve performance. If leaders feel they already have alternative or built-in markets, they 
may pay little attention to such information. Taken together, these properties substantially 
limit the ability of market information by itself to discharge a meaningful quality assurance 
function. 

Combining purposes and policy strategies provides a useful conceptual point of departure for 
more concrete discussions about creating a coordinated system. This occurs partly because it 
helps to highlight particular clusters of alternatives that may be more or less appropriate for 
particular actors to undertake. More important, it emphasizes the ways these strategies can work 
together to help define such a system. Market systems, for instance, are currently fashionable 
because they are consistent with a rhetoric of deregulation, because they fit the diverse (and to a 
significant extent privately controlled) universe of higher education providers and because they 
can respond flexibly to the needs of different constituencies for accountability. But the market 
may be far more effective in influencing behavior in some sectors than others and may be 
extremely uneven in assuring quality about things that are important to some constituencies. 
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Here the use of regulatory mechanisms and incentives becomes salient, not to replace the 
operation of the market, but to shape it in particular ways. State regulation, for example, can be 
used to ensure that basic consumer information is in fact present. This was a basic theme in 
Colorado Governor Roy Romer's Quality Counts agenda when he chaired ECS in 1994-95. 
Incentive systems operated by both the states and the federal government, moreover, allow these 
actors to enter the market on their own behalf, in pursuit of public purposes, to ensure that their 
own interests in accountability are met, or in contracting with institutions to deliver certain 
services. An approach that consciously orchestrates incentive-based and regulatory strategies on 
the part of all accountability players seems much better suited to addressing such complex 
matters as access, cost and choice, and thus seems superior to any purely market-driven or 
regulatory alternatives. 

Actors and Assignments in a Coordinated System 

The extent and effectiveness of any future accountability system will depend decisively on the 
motives, resources and constraints of the various sectors involved. The conditions that led to the 
1992 Amendments, as well as attempts to implement its provisions from 1992 to 1994, provide 
ample evidence that members of the triad cannot or will not act in certain ways, regardless of the 
theoretical merits of their assigned roles. At the same time, other agents beyond the traditional 
triad have parts to play. These agents include institutional governing boards, and disciplinary and 
professional associations, as well as the "marketplace" and the third-party information-providers 
that inform it. For each component of the proposed system, it is important to recognize the basic 
interests, available strengths or resources, and important operational constraints that delineate 
what each agent does best. Only with this as a foundation can specific suggestions be made 
regarding what role each of these agents should play within a coordinated system of quality 
assurance. Many of these assignments are congruent with existing practices. But clarifying the 
important differences among these assignments is a necessary first step in building consensus for 
a refashioned approach. 

The Federal Government 



Basic federal interests in accountability include ensuring that funds expended through federal 
programs are not misappropriated and allocating funds to support intended policy purposes. 
Historically, the federal focus has been on broadening access to postsecondary opportunities, 
largely to help provide economic benefits to individuals. Federal interest in such matters as 
consumer protection and establishing minimum standards of institutional capacity for 
participation in federal programs, as well as its effort to limit individual participation to those 
with an "ability to benefit," all follow from this basic policy motivation. These matters have little 
to do with academic quality per se. 

This circumscribed policy interest is a major strength of the federal government because it 
potentially allows national regulatory efforts to be focused and limited. Another unique strength 
of the federal position is its ability to establish common definitions for performance measures. 
This role was demonstrated convincingly (if unintentionally) by the history of "Student 
Right-to-Know." In 1990, federal guidelines to implement the "Student Right-to-Know and 
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Campus Security Act" mandated a particular approach to reporting graduation rates. Although 
these initial guidelines were officially withdrawn in response to an angry backlash, by 1996 most 
states and a number of institutions had instituted them voluntarily in their own reporting 
practices. 

Through its requirement that participation in the Integrated Postsecondary Education Data 
System (IPEDS) be a condition for continuing institutional eligibility for federal funds, the 
federal government can leverage development of common definitions and calculation procedures 
for performance statistics such as like graduation rates and employment placement rates. Its 
direct interest in this matter, as well as its central position, means that the U.S. Department of 
Education, operating through such agencies as the National Center for Education Statistics 
(NCES), is in the best position to conduct such activities nationally. Finally, the federal 
government provides an excellent mechanism for sponsoring policy discussions and research, the 
results of which can be used to clarify emerging policy disputes. Both its information gathering 
and research functions potentially allow the federal government to assist the "marketplace" by 
ensuring integrity of market information. 

Weaknesses of the federal position include inertia, distance from the action and heavy constraints 
on its ability to establish different rules for different players. SPRE was an attempt to overcome 
some of these conditions by actively working with states to allow them to develop their own 
rules within federal guidelines. A major flaw in SPRE, however, was that it attempted to directly 
replicate federal approaches at the state level rather than helping states capitalize on what they 
do best. For example, the highly regulatory approach ultimately adopted in SPRE gave states 
little flexibility to evolve institutional review and enforcement mechanisms consistent with their 
existing political cultures and postsecondary oversight mechanisms. More important, these 
regulations required the same types of structures for states that visibly had a fraud and 
malfeasance problem as for those that did not. But the original intent of SPRE, to capitalize on 
state-level familiarity with the institutions within their boundaries and allow states to determine 
the specific types of institutions upon which to focus regulatory attention, was explicitly crafted 
to respond to known weaknesses in the federal government's ability to undertake key 
accountability tasks. 

Given these strengths and weaknesses, the federal government's role in a coordinated system 
should address the following: 

* Act directly to eliminate fraud and malfeasance in Title IV programs. Such action is in 
the direct interest of the federal government. But initiatives designed to fulfill this 
enforcement function should be focused narrowly and specifically on the parameters of the 
problem itself and should be applied proactively where specific instances of fraud and 
malfeasance are found. Unannounced visits and seizure of records may be appropriate tools 
within this function, so long as their scope is limited to Title IV enforcement. 

• Ensure that participating institutions are able to administer federal programs. A 

slightly broader, yet consistent, federal interest follows from its role as a contractor for 
institutional administrative services, principally those connected with Title IV. The 
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establishment and enforcement of minimum threshold standards with respect to fiscal and 
administrative capacity is completely appropriate here. It is in the federal government's 
interest to engage in and continue the implied contractual relationship of administering 
federal programs for institutions that meet such standards, and to discontinue the 
participation of those that do not. This standard-setting function by the federal government 
couples nicely with the quite different quality-assurance missions of states and accreditors, 
which might more visibly reference these threshold standards as appropriate in their own 
processes. At the same time, these actors could concentrate more particularly on areas not 
related to the purely administrative and fiscal matters already addressed by federal reporting 
and review. Such an integration would minimize process duplication and wasted effort at 
postsecondary institutions. Furthermore, rather than examining and setting standards for 
educational outcomes, the federal government's involvement in oversight would be limited 
to an earlier and more appropriate stage. 

Take leadership in setting common standards for information including definitions, 
methodologies for data collection and standard formats for communicating 
institutional performance. Both the central position and considerable existing capabilities 
of such agencies as NCES suggest that this role is uniquely federal. States and individual 
accreditors are not in a good position to take the lead in setting such standards. But both 
might collaborate usefully with the federal government to develop and commonly adopt 
clear conventions to be used nationally for calculating and reporting information about 
performance. Each agency then could require appropriate use of these common statistics 
within its own auspices. 

The recent work of the Joint Commission on Accountability Reporting led by the American 
Association of State Colleges and Universities and the still-to-be-finalized activities of 
NCES' National Postsecondary Data Cooperative, provide useful opportunities to further 
this agenda. In support of these agencies' efforts, the federal government can continue to use 
Title IV eligibility as an ultimate regulatory lever to ensure that institutions adopt these 
widely accepted conventions. But in doing so, the common goal should be the establishment 
of mutually beneficial national information standards, not simply federal reporting 
requirements. 

Take the lead in facilitating information sharing among all relevant agencies 
regarding fraud, malfeasance and institutional administration of federal programs. 

Again, enforcement in these matters is a direct federal interest, suggesting that the federal 
government should take the lead in establishing and maintaining the necessary lines of 
communication. In particular, this leadership should be directed toward improving 
coordination among states, accrediting agencies and regional offices of the U.S. Department 
of Education in conducting institutional reviews. Specific initiatives that might be explored 
here include joint participation in certain types of reviews and establishment of a common 
database of institutions containing limited descriptive and performance-related information 
that could be accessed and updated by all parties. 
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States 



State interests in accountability are myriad and in many ways contradictory as well. First, states 
are the direct owners and operators of the significant share of postsecondary education 
represented by public colleges and universities. This essentially proprietary interest means that 
states must be directly concerned with basic operational efficiency and integrity, return on 
investment and appropriate stewardship of core assets for these institutions. Second, states have a 
less immediate, but still direct, interest in building an effective postsecondary system, one that 
promotes broad-based economic competitiveness and improved quality of life within the state. 
This role goes beyond proprietary interest to embrace the perspectives of employers and 
individual citizens. Third, states retain some interest in protecting their citizens against fraud, but 
no more so in higher education than in any other business. Although all three areas are related to 
outcomes, the depth and vigor of state engagement in matters of accountability varies 
substantially across these three interests, and most states, quite understandably, concentrate their 
efforts on the first. 

A major strength of the states from an accountability standpoint is their potential role in setting 
clear public purposes for postsecondary education within their jurisdictions and in determining 
the degree to which these purposes are met. State control of the financial affairs of a large 
number of institutions, as well as other assets that may affect all providers such as student 
assistance or access to technology resources, is another advantage. As a result, states are 
potentially far more able to leverage institutional behavior than are most other external actors, 
though they rarely make full use of this potential. Finally, states play an important role in 
coordinating all the higher education assets (public and private) that lie within their boundaries. 
This role in many cases involves licensing and approving programs operated by nonpublic 
institutions or providing these institutions with public funds under various circumstances (e.g., 
capitation grants or state-level student assistance programs). 

Because of differences in political structure and culture, though, states always will vary in their 
interest and capacity to do much more than act as the proprietors of public institutions. Some 
may actively embrace "gatekeeping" and consumer protection in education as they do in other 
enterprises. Others similarly will promote easy entrepreneurship and caveat emptor. This is not a 
condition that either federal regulation or lobbying on the part of various postsecondary 
education constituencies is likely to change. 

At the same time, partly as a result of the SPRE experience, more and more states appear 
interested in strengthening their gatekeeping functions, and a majority are more familiar than they 
once were with proprietary institutions operating within their boundaries. In general, both 
conditions mean that state-level action within this domain is likely to be more effective than in 
the past. In some cases, moreover, the political linkages between states and their major 
postsecondary institutions may be so intimate that they effectively constrain the state's ability to 
regulate them. This condition is ironic because these are the institutions that the states nominally 
own and control. Finally, although close contact between states and institutions constitutes an 
informational asset, the same intimacy also entails greater political entanglement. In short, 
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though they know more, state agency officials may be less able than other players to act on what 
they know because they must live and deal with local interests on an ongoing basis. 

Given these attributes, the optimal role for the states in a coordinated approach to quality 
assurance might include: 

• Ensure direct accountability for public institutions. Because states are the owners and 
operators of a considerable postsecondary asset, they are in a special position with respect to 
accountability for these institutions. In particular, state officials require assurance that 
public financial resources are being used appropriately, and they are bound to distribute 
information on the return on investment associated with this support. Unlike the federal 
government, states have a direct interest in examining matters of public institutional 
performance. But, states vary significantly in both their interest and capacity to develop 
such reporting mechanisms. Therefore, it would be useful for states to adopt common 
measurement and reporting conventions, and to collaborate actively with proposed federally 
led initiatives designed to establish common definitions and methodologies. 

• Foster the overall development of the state's higher education asset consistent with 
societal need. As mentioned previously, states also have a legitimate interest in ensuring 
that higher education assets of all kinds — public, private and proprietary — are distributed 
effectively, not only to provide access to citizens but also to meet state economic and social 
needs. To ensure this alignment, states may use both regulatory mechanisms (e.g., program 
approval) and incentives (e.g., capitation grants or direct aid to individual institutions). For 
accountability purposes, however, the primary interest should be centered on system 
performance rather than on individual institutions' performance. A possible exception is 
where states run their own student aid programs. In such cases, state interest parallels that of 
the federal government, and state actions should be coordinated directly with federal 
activities to maximize information and minimize duplication. 

• Ensure basic levels of consumer protection. Except in cases of interstate commerce 
which fall under federal jurisdiction, consumer protection is a basic state interest. At a 
minimum, consumer protection in higher education requires two functions. The first is 
periodic certification that a given education provider actually exists and is delivering 
instruction essentially as advertised. This certification requires both initial licensure and 
some form of ongoing monitoring, with the primary focus of both being to establish and 
enforce "truth in advertising." 

Extending this function might involve ensuring that institutions provide potential 
consumers with useful and accurate information for making choices. A second state-level 
consumer protection function is to collect and follow up on consumer complaints. This 
function need not be specific to education providers and should be consistent with the 
requirements associated with any business. Within both functions, states legitimately and 
inevitably will vary in their level of prescription. But it seems appropriate that a set of 
baseline standards be evolved through agencies such as ECS or SHEEO. 
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Institutional Accreditors 



The basic interest of institutional accrediting bodies lies in protecting the integrity of the 
academic enterprise as a whole by setting standards that differentiate "reputable" institutions 
from other schools whose "disreputable" operations might damage the credibility and standing of 
all. In essence, accrediting associations are voluntary bodies whose individual institutional 
members agree to give up total autonomy and to subject themselves to peer review. In return, 
they obtain a credential of some value in the student and fiscal marketplace and protection 
against the more blatant claims of unscrupulous competitors. As voluntary, provider-owned 
organizations, accrediting bodies have only an indirect interest in ensuring that public needs for 
postsecondary education are met, in assisting consumer choice, or in examining particular levels 
of performance or investment returns for either funders or individuals. 

Because they act in response to the interests of institutional members, an important strength of 
accrediting bodies is their perceived legitimacy across institutional sectors. Their standards for 
recognition are centered on issues important to academics and members of the academy familiar 
with academic operations conduct the review processes to determine whether these standards are 
being met. As a result, significant institutional investments (at levels often surprising to 
outsiders) are made in the accreditation process, even by those institutions with little to fear 
about its outcome. A related strength of institutional accreditation is the peer review process 
which constitutes its core. Expert judgment is arguably the only effective way to examine more 
esoteric dimensions of academic performance; when the process works well, such judgments can 
be both strict and credible. At the same time, the considerable collaboration which the peer 
review process entails — if it is well focused — can have substantial benefits in improving 
institutional practice. 

The primary disadvantages of accrediting bodies as accountability players are that they are 
unlikely to sanction their members frequently or severely, are unable to engage in obtrusive or 
aggressive data-gathering and investigative activities, and tend to limit the extent to which 
findings are publicly distributed. By its very nature as an "inside" operation, moreover, 
accreditation always will have credibility problems for outside constituencies. Finally, 
accreditation’s central mechanism of peer review has all the strengths and drawbacks of an 
owner-operator activity. On the one hand, it enjoys internal legitimacy and is usually conducted 
with energy and effect by respected colleagues committed to the enterprise. On the other hand, 
the standards of evidence used in reviews can be idiosyncratic, the training in observation and 
evaluation received by peer reviewers uneven, and review documentation inconsistent. 

Given these strengths and weaknesses, institutional accreditors should have the following roles in 
a coordinated system: 

* Set and communicate standards about institutional integrity. While federal and state 
authorities are concerned about fiscal integrity, the interest of accreditors should center 
specifically on the types of integrity peculiar to academic institutions. Most prominently, 
these include ensuring the basic academic freedoms of community members, reasonable 
levels of institutional autonomy (including protecting the role of independent governing 
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boards), and freedom to establish curricular designs and instructional experiences without 
political interference. At the same time, accreditors should ensure that institutional missions 
remain appropriately academic and that institutional resources indeed are directed toward 
educational ends. While both require some consideration of administrative capacity, the 
focus is quite different from the capacity reviews necessary for state licensure or 
participation in federal programs. The latter, for instance, are far more a matter of proper 
accounting than of appropriate internal allocation. 

Set and communicate standards about collegiate attainment. This goal addresses the 
integrity of institutionally awarded degrees and certificates at all levels. As an academic 
matter, institutions acting in concert clearly should be responsible for certifying a degree's 
academic content. But doing so is probably also the single biggest challenge that accreditors 
need to address. At a minimum, accrediting bodies need to develop a framework of criteria 
for analyzing collegiate curricula that is much more focused on academic achievement than 
is the case for current accreditation standards. Extending this framework of criteria 
eventually might entail specifying the content that ought to be contained in areas such as 
baccalaureate general education. Finally, establishing such a framework also demands a 
continuing emphasis on refining an institution's own internal academic assessment and 
quality-assurance processes as emphasized in a 1996 study by Dill, Massy, Williams and 
Cook. 

Stimulate institutions to improve and communicate "quality" to outside audiences. 

While the first two goals emphasize standard-setting and summative judgment, this goal 
emphasizes continuous improvement. To be maximally effective, "quality" discussions 
should emphasize outcomes and the value added by education. A primary concern should be 
to stimulate institutions to increase their capacity and sophistication in these areas. Because 
working toward "quality" in this sense is a continuous function, greater consideration might 
be given to making the consultation process associated with accreditation less episodic than 
the current self-study/review cycle appears to be. Again, the voluntary adoption by all 
institutional accrediting bodies of a common set of reporting statistics aligned with national 
guidelines and state requirements, and the use of these statistics in more frequent 
institutional reporting, would be particularly helpful in gamering legitimacy for 
self-regulation among outside constituencies. 

Ensure the integrity and alignment of the various non-governmental 
quality-assurance processes. Finally, accreditors themselves should evolve a clearer set of 
common purposes and should reestablish collective, nongovernmental certification 
processes that ensure that these purposes are met by each accrediting agency. In addition, 
teams drawn from all accrediting bodies should strive to achieve greater consistency in 
applying standards and review procedures across all accrediting organizations and regions. 
The newly established Council for Higher Education Accreditation (CHEA) shows initial 
signs of pursuing these twin objectives — though the former more visibly than the latter. 
Ultimately, however, both will be important if accreditation is to retain its current 
prominent place at the accountability table. 
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Institutional Govemine Boards 



Though not explicitly recognized as national quality-assurance players, institutional and system 
governing boards have many interests related to accountability. First, whether public or private, 
such boards are charged with a basic accountability responsibility. In the public sector, this 
responsibility is especially salient and often highly detailed, though it has been chiefly exercised 
in fiscal affairs. Boards also have an explicit responsibility for ensuring the continuing viability 
of their institutions. With respect to public accountability, however, this responsibility has both 
advantages and disadvantages. On the one hand, boards may be directly interested in information 
about quality in order to identify and address problems that may affect long-term institutional 
health. In the short run, however, boards may resist the release of any information seen as a threat 
to the institution or that portrays it in a less-than-favorable light. 

Major governing board strengths in the realm of accountability are their direct knowledge of 
institutional conditions and their ability to do something about them. Major weaknesses are that 
they frequently lack knowledge about (or an interest in) nonfinancial matters. Looked at 
collectively, governing boards also are relatively isolated from one another and consequently 
have little ability to develop consistent approaches to problems or issues across institutions (a 
condition less true of multi-institutional public boards). Finally, as might be expected, 
institutional governing boards have an undeniably parochial interest in their own institutions. 

Despite their absence from the traditional triad, institutional governing boards do have important 
roles to play in a refashioned national system of accountability. These activities might include: 

• Ensure direct fiscal and administrative accountability for institutions. In the case of 
public institutions, boards act as direct agencies of the state in discharging fiscal and 
administrative functions, and this function should be respected and reinforced. In the case of 
private institutions, boards also assume the same scope of responsibility for accountability 
as previously delineated for the state's responsibility in terms of public institutions. As a 
result, they have an obligation to report regularly and consistently to their stakeholders on 
the condition of the enterprise. 

• Determine institutional mission and ensure it is being pursued effectively. At both 
public and private institutions, ensuring that the institution remains true to its mission is a 
major governing board function. But considerable attention to assessing and communicating 
institutional effectiveness at the broadest level is required to discharge this obligation. Like 
fiscal accountability, though, governing boards should limit their focus to tracking 
outcomes and should not intrude into internal academic management and decisionmaking 
processes. One method for achieving both this and the previous goal, as noted earlier, might 
be to evolve an academic equivalent of a corporate stockholder's report for each institution. 
Like a fiscal audit report, such a document could be used to support both accreditation and 
government-run reviews. 

• Ensure the continuing viability of the institution by containing costs and increasing 
productivity. This responsibility lies explicitly with the board. If it is discharged effectively 
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and if results are communicated clearly to the public, this responsibility has important 
consequences and is useful in a broader system of quality assurance. Again, though, clear 
and consistent communication is the key. 

Disciplinary and Professional Organizations 

Arguably the most powerful players in terms of quality assurance in the strictly academic arena, 
disciplinary and professional organizations exert an influence that cuts across that exercised by 
members of the traditional triad. In the form of specialized accrediting bodies, they have 
substantial leverage on departmental, and hence collective institutional, behaviors. Unlike those 
of institutional accreditors, the actions of these bodies generally confer a credential that is of 
considerable value in the marketplace. The influence of more traditional disciplinary associations 
is less immediate but remains potentially important. In both cases, these associations are where 
the vast majority of faculty owe their allegiance, often far more so than to the institutions that 
employ them. 

In the case of specialized accreditation, basic motivations are those of any trade association — to 
protect the interests of its members. This protection is achieved indirectly by ensuring the 
integrity of the profession or discipline and more directly, by controlling access to and leveraging 
greater support for it. In the case of more traditional disciplinary associations, basic motivations 
are to provide individual faculty members with opportunity to further their own research and 
career paths and, through peer review, to help maintain the integrity of scholarship in the 
discipline. 

Consistent with these purposes, major strengths of professional and disciplinary bodies are their 
direct alignment with some faculty interests and their ability to allocate status. These strengths 
make it likely that these organizations will be respected internally. Major weaknesses are their 
narrowness of focus, their fragmentary coverage of institutional offerings and activities when 
taken collectively, and their lack of coordination. The first and second mean that critical 
cross-cutting concerns such as the quality of the undergraduate experience or the integrity of 
institutional operations may not be attended to at all. The third, at least in the case of specialized 
accreditation, means that institutions are approached piecemeal to engage in multiple, 
time-consuming and highly duplicative review processes, often resulting in information 
unavailable to the public. Finally, of course, specialized accreditation bodies are as open to 
charges of uneven quality in the review processes they operate as their institutional counterparts. 

Disciplinary and professional organizations, of course, already play a prominent role in assuring 
quality, but this role is currently unsystematic. Among the functions that might be explicitly 
assigned to such organizations under a coordinated national approach are: 

* Share information and disciplinary expertise directly with institutional accreditors to 
increase quality and reduce the institutional burden of peer review. The largest 
universities must assemble data for and undergo peer review by up to several dozen 
professional accreditors. And because the possibility of denial is real and its consequences 
significant, these largest institutions may spend more total resources on such processes than 
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on the far more comprehensive ones run by institutional accreditors. Increased 
coordinationbetween regional and special accreditors — and in particular, the possibility of 
conducting joint reviews — could pay substantial dividends for institutions in reducing 
burden, yet could still allow independent judgments to be made about the quality of specific 
disciplines and programs. 

Joint activities of this kind also might allow institutional accreditors to pay greater attention 
to academic issues that cut across the institution — such as general education or the 
meaning of the baccalaureate — than they currently do. Cooperative reviews, moreover, 
might take place for all institutions offering a particular discipline, regardless of whether or 
not they are seeking recognition by a particular specialized body. More radically, regional 
accreditors might seek to involve traditional disciplinary associations to help assess parts of 
the curriculum not currently subject to special accreditation. 

• Adopt common approaches for collecting data and reporting performance. As noted 
earlier, duplicative data collection and reporting — and more important, the fact that each 
professional accreditor may have its own way of counting — confuses external audiences 
and increases institutional burden. At a minimum, all such organizations voluntarily might 
adopt uniform approaches to data collection and a single point of data reporting to serve 
collective needs under the auspices, as noted, of a national effort to develop such standards. 

* Avoid specific linkages between specialized accreditation and institutional eligibility. 

As a result of the "contractual" relationship noted earlier, the federal government has an 
interest in ensuring institutions meet minimum standards of financial and administrative 
capacity. Beyond this, both federal and state authorities are interested in assurances that 
minimum levels of general academic quality are being maintained, a job legitimately 
delegated to regional accreditors. Specialized accreditation, in contrast, may have 
marketplace value for an institution, but focuses on matters basically unrelated to 
stewardship of government funds. This distinction between the respective roles of different 
accrediting bodies should be strictly maintained in any restructuring of the current system. 

Third-party Information Providers 

A range of additional actors that rate institutions in various ways fills a major functional vacuum 
in the national approach to quality assurance — that of informing consumer choice. Prominent in 
this group are college guides and national magazines. Their basic interests are purely 
commercial: to sell publications in response to evident consumer demand. To serve this interest 
effectively, however, such organizations must adequately assess demand and continue to provide 
information large numbers of people find useful. Despite their many critics inside the academy, 
the majority of these providers clearly have met this demand in a way that no one else has. 

Other third-party information providers represent special interests within higher education that 
exercise a self-regulatory function outside the realm of academic quality — for example, the 
National Collegiate Athletic Association (NCAA). Basic interests here, though more narrowly 
focused, are similar to those exercised by specialized accreditation. Because of their focus on 
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statistical performance reporting, however, such organizations already may be recognized 
formally in legislation or regulation (e.g., the NCAA in "Student Right-to-Know"). 

The level of recognition and market response experienced by such third parties is arguably both a 
strength and a weakness for accountability. On the one hand, they likely will be sensitive to the 
kinds of information being asked for by special interests and by consumers of higher education. 
On the other hand, they likely will do the minimum needed to meet perceived customer needs 
and, in fact, may attempt to shape the content of such demands for information in the first place 
— especially when they actively represent such special interests. Added to this disadvantage is 
the fact that third-party information providers tend to focus disproportionate attention on the 
market's "high end" where consequential choice making happens and where consumers invest in 
the type of information provided. As a result, mechanisms based solely on market information 
may serve eventually to drive improvement only for certain types of institutions, while ignoring 
the need to provide basic consumer protection in other sectors. 

Given these strengths and weaknesses, third-party information providers still have a part to play 
in accountability and might explicitly be recognized as serving the following roles: 

• Repackage standard, agreed-upon information about institutional performance in 
ways suited to the needs of particular consumer and constituency groups. National 
magazines and consumer guides already have expressed a desire to work with colleges and 
universities to improve the ways they collect and report data. In fields other than higher 
education, they often use standard performance measures developed by others, as examples 
ranging from EPA mileage ratings for automobiles to FAA-provided on-time ratings for 
airlines attest. Controlling access to such data, once developed, could provide a means to 
leverage private information providers toward more responsible reporting. Eligibility to 
access common databases could be made conditional on periodic audits of the use made of 
such information, conducted by accreditors and/or other designated representatives of the 
higher education community. 

• Conduct market research to determine what the different consumers of higher 
education really want to know. The higher education community routinely complains that 
third-party information providers oversimplify quality by taking a purely statistical 
approach. But academics also fail to recognize that publishers are reacting to clear demands 
from their readers for such information. At the same time, it is fair to say that much of the 
information reported by such providers falls short of what different kinds of customers 
really want to know. Especially lacking in the current information marketplace is 
performance data related to the needs of students in the new majority — students who are 
non traditional, financially limited and tied to a particular location. National magazines do 
not rate community colleges and proprietary schools. Third-party information providers, of 
course, will require persuasion that a viable market for such information exists. But if this 
can be established, it is very much in the interest of such organizations to determine what 
this new majority really wants to know about colleges and universities — and what they 
will pay to find out. 
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The Market 



Given current conditions, accountability functions not specifically assigned to one or more of the 
actors above should be largely evolved through the continuing operations of the market. Among 
these functions are: 

• Stimulate the development of constituency-specific sources of consumer information. 

Third-party providers and institutions themselves constitute the best source of this 
information, motivated as they are by current market position. As noted, key accountability 
players, under the leadership of the federal government, should collaborate in developing 
clear standards of reporting and mechanisms for ensuring "truth in advertising.” But the 
provision of detailed consumer information should not become a core function of either 
state or federal governments or of institutional accrediting bodies. 

• Help contain costs. For public institutions, of course, state governments have a direct role 
to play in containing costs. As tuition's share in typical public institutional cost structures 
continues to rise, however, the market will play a greater role. For private institutions, cost 
always has reflected market mechanisms. For both kinds of institutions, the need to ensure 
cost-effectiveness has proved far more powerful in stimulating internal improvement and 
subsequent efforts to communicate return on investment to potential customers than has any 
form of government action. Again, however, the appropriate role of government or 
accreditors may be to help develop the definitions and methodologies by which such 
information is collected and communicated. 

These suggested roles constitute a concrete starting point for further discussion and are in no way 
intended to be final. This paper has not attempted, for example, to prescribe such roles 
extensively because exploratory conversations about this topic have just begun and so far have 
included only representatives drawn from the traditional triad. But if some degree of consensus 
can be built around suggested basic purposes and first principles, additional elaboration of these 
potential assignments and action steps is warranted. 
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SUMMARY 



The premise that a new national approach to accountability and quality assurance in higher 
education is needed rests on a growing conviction that the present system is both ineffective and 
unwieldy. The increasing diversity and complexity of higher education as a "mature" industry, to 
use Arthur Levine's characterization in a January 1997 article in the Chronicle of Higher 
Education, as well as the growing unworkability of government-imposed solutions to social 
problems in all sectors, demands a more decentralized approach. 

The coordinated system of quality assurance suggested in this paper rests fundamentally on the 
proposition that different parties-at-interest have different needs, resources, motivations and 
limitations when discharging accountability functions. It insists, moreover, that each of these 
perspectives is legitimate, and that all can be harnessed as part of a refashioned approach. Doing 
so, however, does not mean leaving each agency to its own devices. Clear consensus is required 
to establish practical boundaries for the agencies’ respective roles. Substantial agreement also is 
needed about definitions of quality and the measures that make them real. This conversation will 
not be easy, but whatever its eventual outcome, it is much warranted. 
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